Combined GKLT Feature Tracking and Reconstruction for Next Best View Planning

Guided Kanade-Lucas-Tomasi (GKLT) tracking is a suitable way to incorporate knowledge about camera parameters into the standard KLT tracking approach for feature tracking in rigid scenes. By this means, feature tracking can benefit from additional knowledge about camera parameters as given by a controlled environment within a next-best-view (NBV) planning approach for three-dimensional (3D) reconstruction. We extend the GKLT tracking procedure for controlled environments by establishing a method for combined 2D tracking and robust 3D reconstruction. Thus we explicitly use the knowledge about the current 3D estimation of the tracked point within the tracking process. We incorporate robust 3D estimation, initialization of lost features, and an efficient detection of tracking steps not fitting the 3D model. Our experimental evaluation on real data provides a comparison of our extended GKLT tracking method, the former GKLT, and standard KLT tracking. We perform 3D reconstruction from predefined image sequences as well as within an information-theoretic approach for NBV planning. The results show that the reconstruction error using our extended GKLT tracking method can be reduced up to 71% compared to standard KLT and up to 39% compared to the former GKLT tracker.